Papers with autoregressive generation
Non-Autoregressive Sequence Generation (2022.acl-tutorials)
Copied to clipboard
| Challenge: | Non-autoregressive sequence generation (NAR) models generate output sequences in parallel to speed up generation process. |
| Approach: | This tutorial provides a thorough introduction and review of non-autoregressive sequence generation . it aims to generate the entire or partial output sequences in parallel to speed up the generation process . |
| Outcome: | This tutorial provides a thorough introduction and review of non-autoregressive sequence generation . it aims to reduce the performance gap between state-of-the-art models due to lack of modeling power . |
Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generation (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing non-AutoRegressive (NAR) text generation models lack proper pre-training, making them far behind pre-trained autoregressive models. |
| Approach: | They propose a novel pre-training task to promote prediction consistency in non-autoregressive (NAR) generation. |
| Outcome: | The proposed model outperforms existing pre-trained models and achieves 17 times speedup in throughput. |
Diffusion Language Model Inference with Monte Carlo Tree Search (2026.findings-eacl)
Copied to clipboard
Zheng Huang, Kiran Ramnath, Yueyan Chen, Aosong Feng, Sangmin Woo, Balasubramaniam Srinivasan, Zhichao Xu, Kang Zhou, Shuai Wang, Haibo Ding, Lin Lee Cheong
| Challenge: | Existing methods for inference use heuristics to determine which positions to unmask and which tokens to commit . MEDAL is an inference-time scaling framework that integrates Monte Carlo Tree SEarch initialization for Diffusion Language Model inference. |
| Approach: | They propose a framework that integrates Monte Carlo Tree SEarch initialization for Diffusion Language Model inference. |
| Outcome: | The proposed framework achieves 22.0% improvement over existing inference strategies across multiple benchmarks. |
Generative Models for Automatic Medical Decision Rule Extraction from Text (2024.emnlp-main)
Copied to clipboard
| Challenge: | Medical decision rules are traditionally constructed by medical experts, which is expensive and hard to scale up. |
| Approach: | They propose to extract medical decision rules from text using generative models . their code will be open-source upon acceptance . |
| Outcome: | The proposed model outperforms state-of-the-art models on a Chinese benchmark and achieves 67% tree accuracy. |
POS-Constrained Parallel Decoding for Non-autoregressive Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing non-autoregressive generation systems face multimodality problem due to conditionally independent decoding. |
| Approach: | They propose to incorporate linguistic structure into NAG inference instead of teacher AG . they propose a method that provides a specific POS sequence to constrain the NAG model . |
| Outcome: | The proposed method improves NAG models on four text generation tasks to a greater extent compared to knowledge distillation. |
OjaKV: Context-Aware Online Low-Rank KV Cache Compression (2026.findings-acl)
Copied to clipboard
Yuxuan Zhu, David H. Yang, Mohammad Mohammadi Amiri, Keerthiram Murugesan, Tejaswini Pedapati, Pin-Yu Chen
| Challenge: | Existing methods for inference use static, offline-learned subspaces that perform poorly under distribution shifts. |
| Approach: | They propose a framework that integrates a storage policy with an online subspace adaptation to preserve key-value tokens in full rank as high-fidelity anchors. |
| Outcome: | Experiments show that OjaKV maintains or improves zero-shot accuracy at high compression ratios, achieving the strongest gains on long-context benchmarks requiring complex reasoning. |
Finetuning Pretrained Transformers into RNNs (2021.emnlp-main)
Copied to clipboard
Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A. Smith
| Challenge: | Efficient transformers outperform recurrent neural networks in natural language generation, but this comes with significant computational cost and memory footprint during generation. |
| Approach: | They propose to convert a pretrained transformer into its efficient recurrent counterpart, improving efficiency while maintaining accuracy. |
| Outcome: | The proposed transformers outperform recurrent neural networks in natural language generation but come with significant computational and memory footprint during generation. |
Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Speculative decoding (SD) is a powerful and efficient way to accelerate autoregressive generation. |
| Approach: | They propose a training-free framework that recovers valid tokens discarded by standard verification . they use online correction memory and Semantic Consistency Gating to analyze rejections . |
| Outcome: | The proposed framework outperforms existing methods and achieves peak throughput speedup of 2.33x. |
Multi-view-guided Passage Reranking with Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models rely on autoregressive generation and sliding window strategies to rank passages, which incur heavy computational overhead as the number of passages increases. |
| Approach: | They propose a non-generative LLM-based reranking method that encodes query-passage information into diverse view embeddings without being influenced by external biases. |
| Outcome: | The proposed model matches the performance of much larger 7B-scale fine-tuned models while achieving a 100x reduction in inference latency. |
DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Speculative decoding (SD) has proven to be effective for autoregressive generation in large language models (LLMs), however its application to vision-language models (VLMs) remains relatively unexplored. |
| Approach: | They propose a Speculative Decoding framework for vision-language models that integrates a neural architecture search framework and target-aware supernet training to identify optimal interaction strategies. |
| Outcome: | DREAM-S achieves 3.85 speedup compared to baselines on well-established vision-language models. |